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Abstract 

For an indecomposable 3x3 stochastic matrix (i.e., 1-step transition probability 
matrix) with coinciding negative eigenvalues, a new necessary and sufficient 
condition of the imbedding problem for time homogeneous Markov chains is 
shown by means of an alternate parameterization of the transition rate matrix 
(i.e., intensity matrix, infinitesimal generator), which avoids calculating matrix 
logarithm or matrix square root. In addition, an implicit description of the 
imbedding problem for the 3x3 stochastic matrix in |13| is pointed out. 
keywords: imbedding problem; three-state time homogeneous Markov chains; 
negative eigenvalues 
MSC numbers: 60J10; 60J27; 60J20 
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1 Introduction 

The imbedding problem for finite Markov chains has a long history and was 
first posed by Elfving [5], which has applications to population movements in 
social science [21j . credit ratings in mathematical finance jllj . and statistical 
inference for Markov processes [TJ [TS] . For a review of the imbedding problem, 
the reader can refer to [2]. 

According to Kingman I16j , the imbedding problem is completely solved 
for the case of 2 x 2 matrices by D. G. Kendall, who proved that a 2 x 2 transition 
probability matrix is compatible with a continuous Markov process if and only 
if the sum of the two diagonal entries is larger than 1. 

The explicit description of the imbedding problem for the 3x3 stochastic 
matrix with distinct eigenvalues or with coinciding positive eigenvalues is shown 
by Johansen in |13j . When the common eigenvalue is negative, P. Carette 
provides several necessary and sufficient conditions to characterize the imbedded 
stochastic matrix ^ Theorem 3.3, Theorem 3.6]. But in contrast to the above 
conclusions of Kingman or Johansen, it is not clear-cut. 

Let I be the identity matrix. Let P, P°° be the 3x3 stochastic matrix and 
its limiting probability matrix respectively. Theorem 3.3 ( rcsp. Theorem 3.6,) 
of [2] needs to calculate square root of the matrix P°° — I (rcsp. P), and then to 
test whether each of the off-diagonal elements satisfies an inequality. But the 
matrix square root is many- valued, just like the matrix logarithm |21[ p22]. 

In the present paper, a new necessary and sufficient condition is shown, 
which overcomes the difficulty of uncountably many versions of logarithm or 
square root (Theorem 12.61) . We chiefly rely on an alternate parameterization of 
the transition rate matrix (Eq.tJBJ)) for the proof. At the same time, for a fixed 
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P°° , the exact lower bound of the eigenvalue of P that makes P embeddable is 
given (Remark [5]L. 

In the more general context of time-inhomogeneous Markov chains, the 
imbedding problem is dealt with by some authors [5J [7J [51 [HI Hlj- But we 
only focus on the time-homogeneous Markov chains here. 



2 The imbedding problem for 3-order transition 
Matrix with coinciding negative eigenvalues 

The transition matrix P is called embeddable if there is a transition rate 
matrix Q for which P = e^. 

Let P = (j>ij) be a 3 x 3 transition probability matrix. Suppose P is in- 
decomposable, i.e., its state space does not contain two disjoint closed sets[H 
P17]. Let the unique stationary probability distribution be p! = (/xi, /X2, Hz) 
with fii + ri 2 + H3 = 1- Let e — (1, 1, 1)' 

Lemma 2.1. Suppose P is indecomposable. If Q is a transition rate matrix 
such that P = e hQ , h> 0, then Q has yu'Q = 0. 

Proof. If a distribution v has ^'Q = (i.e., the left eigenvector with eigenvalue 
0), then P = e hQ = J2™=o ^"Q™ implies that v'P = v' . Since P is indecom- 
posable, one has that v = fx. □ 

Lemma 2.2. Suppose the transition matrix P is embeddable with eigenvalues 
{1, A, A}. A < 0. Then P is diagonalizable and satisfies 

P = P°° + A(l - P°°), (1) 

lr The existence of the lower bound is shown in [2] Theorem 3.7] 
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where P°° = e/i' is the limiting probability matrix, I is the identity matrix\\ 

Proof. If Q is a transition rate matrix such that P = e^, then Q has a pair 
of conjugate complex eigenvalues and is diagonalizablc. Since P = e^, P is 
diagonalizable too. 

The eigenvalues of P are {1, A, A}, thus the rank of the matrix Al — P is 1. 
Since (Al — P)e = (A — l)e, the three rows of Al — P are equal. Then 

1 — (x + y) x y 

z 1 — (y + z) y (2) 

z x 1 — (z + x) 

and 1 — A = x + y + z. Note that // = (z, x, y)/(x + y + z), this ends the 
proof. □ 

Corollary 2.3. Suppose P satisfies the condition of Lemma \2.2\ then the sta- 
tionary probability distribution is positive and all elements of P are positive. 

Proof. Since P = AI + (1-A)P°°, one has that A+(l-A)/^ ^ 0, i = 1,2,3. Then 
l li ^ T^k ^ 0> an< ^ the off-diagonal elements satisfy = (1 — A)/ij > 0, i ^ j. 
In addition, it was shown by Goodman that each of the diagonal elements of 
an embeddable matrix dominates the determinant, and that this determinant 
is positive: p u ^ dct P > [H UM ■ □ 

The conclusions of Lemma 12.21 and Corollary 12.31 also appear in [5] . By 
Lemma 12.21 P is completely determined by its stationary distribution and the 
coinciding eigenvalues. 



2 This conclusion is asserted in 1131 . 
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Proposition 2.4. Suppose that P satisfies Eq.^lty. P can be imbedded if and 
only if there exists a transition rate matrix Q such that it has fi'Q = and 
eigenvalues 9 and 9 and it holds that e = A for some h £ K + . 

The proof of Proposition ^. 41 is presented in Section |2~T1 Here 9 is a complex 



eigenvalue — p + iq with - = 



q _ (2fc+l)7T 
-log|A| 



, k £ Z+. 



Remark 1. By the Runnenberg condition in J2U[5D], the complex eigenvalue 
— p + iq of the transition rate matrix Q satisfies that 



P " V3' 

which the reader can also refer to [3] for detail. Then 



|A| = 0.0043, 



and P is almost equal to its limiting probability matrix by Eq. ([!]). 



For a transition rate matrix 



Q = 



— a 2 — a 3 a 2 a 3 
h -h - b 3 b 3 



Cl c 2 -c\ - c 2 

suppose that it has //Q = 0, that is to say, 



(3) 



Ml a 2 - M2&1 = - M3C2 = M3C1 - Ml«3- 



(4) 



Let 

^ia 2 -/i 2 &i A*ia 2 +/i 2 6i ^3 + u, 3 c 2 /2 3 ci + nia 3 
v= 2 ' 7= 2 = 2 = 2 ' (5) 

We propose an alternate parameterization of the transition rate matrix as 



Q 



k+7 7+1/ k — v 

A»l Ml Ml 



7 — ^ 

M2 



7+<5 S+i/ 

M2 



A»3 M3 



M3 



(0) 
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where 



ft, 7, 6 ^ 0, ft + 7, 7 + (5, <5 + k > 0, and \v\ ft, 7, <5. (7) 

The re-parameterization is one-to-one. Then the transition rate matrix Q with 
H'Q = must satisfy Eq.@. E 

Let the eigen-equation of Q be A(A 2 + a\ + (5) = 0. Then we have that 

K + 7 7 + <5 6 + ft 
a = 1 1 , (8) 

Ml ^2 M3 

^ = ^7 + 7^ + ^ + ^ 

and the eigenvalue is 



= + V/^-" 2 /4- (10) 

Let the ratio between the imaginary (q) and real (— p) parts of the nonzero 
eigenvalues be 



H(k, 7 ,6,v) 4 M 
P 



./ 4 £_ 

a 2 



k7 + 7<5 + ^k + ^ 2 ^ 



+ ^ + ^) 2 

v Ml M3 ' 



Remark 2. Since there may be many different transition rate matrices such 

-1)7T 

PI 



that ff(ft, 7, S, v) = 3^CTxf ' ^ G Z+, the solution of P = e Q is not unique. 



Remark 3. For the given fjf = (fix, {12, (13), if v = and ft : 7 : 6 = : : -i-, 
then i?(/t, 7, (5, 1/) = 0. Note that when 

«;, 7, <5 0, k + 7, 7 + 5, S + k > 0, and |z/| k, 7, <5, (12) 



3 This re-paramctcrization of the transition rate matrix appears in [3], and appears in 
1191 1121 1151 1171 implicitly, which is named the cycle decomposition by some authors. 
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H (k, 7, 8, v) is a continuous real function of («, 7, S, z^)|f| Therefore, H (n, 7, 6, v) 
takes the value _ mjn if the maximum of (k, 7, 8, v) is greater than or equal 



to 



log|A| • 

Therefore, an optimization problem is formulated, and the next Proposi- 
tion solves it. We say that three positive numbers a, b, c satisfy the triangle 
inequality if a + 6 > c, b + c > a, a + c > b. Let 



min{/ii, ^3} • (13) 



Proposition 2.5. Suppose that 



X!X 2 + x 2 x 3 + x 3 xi + (min{xi,x 2 ,x 3 }) 2 
F(x 1 ,x 2 ,x 3 , f i 1 , f i2,^) = (x , +x , X2+x , x , +x ^ v , (14) 

''Ml M2 M3 ' 

m'tt xi,X2,X3 ^ and xi + X2, X2 + x 3 , ^3 + £1 > 0. Then the maximum of 

F(x 1 ,X2,X 3 ,fli,IJ,2,fl3) is 

, ! . i 1 , 1 . 2 , when satisfy the triangle inequality, 

W Ml M2 M3 (15) 

4 ( Y!!^) i otherwise. 
The maximum is attained at the points 

x\ = X2 = x 3 , when --j-, satisfy the triangle inequality, 



^2 = £3 
£3 = £1 





_ _J_ 




1 


CI 


C2 




/''! 






1 
t'l 




2 


1 




1 


/'2 


C3 




/< 1 






1 

C2 




2 


1 




1 


/<;! 


(-'1 




»*2 



Ml M2 M3 ' 



M2 M.3 Ml 



M.3 Ml M2 

(16) 



Proof of Proposition ^. 51 is presented in Section 

Theorem 2.6. Suppose that P is a 3 x 3 stochastic matrix with eigenvalues 
{1,A, A}, A < 0, and that P satisfies Eq.(J^. Then P can be imbedded if and 



4 It is needed that the term inside the *J~- of Eq.JTTJ is positive. 
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— 1 — — , when — , — , — satisfy the triangle inequality, 



(MlM2 + MlM3 + — log I A I ' Ml'M2'M3 

(17) 



or 



m 7r 

„ — : tttj otherwise. (18) 

1 — m - log |A| 

Proof of Theorem \2.b\ Clearly, if the function H (k, 7, 5, v) reaches its maximum 
then v = min{K,7, 5}. From Proposition 12.41 we have to find a transition rate 
matrix Q(k, 7, 8, v) of the form just above with eigenvalues 9(k, 7, <5, v) and 
9(k, 7, (5, 1/) for which e 0h = A for some h > 0. Using Proposition 12 .51 we can find 
(ko, 70, Sq, Pq), so that for the given u! , A we have 

7T 

fl'(«o,7o>^o,t / o) = — ; — jTT = Ho- (19) 
— log |A| 

The corresponding transition rate matrix Q(ko, 70, ^Oj v o) has eigenvalue 0(ko, 70, <5o, v o) 
9q and #cb where 



£o = -f + i^o - all A = f [-1 + iffo] = y[-l + ir^]' ( 20 ) 
We finally choose h = — 2 log |A| /ceo and find 

0„/l = _21og|X| x . 7T 1 

u fto 2 L -log|A|J 

= log |A| + ITT 

which satisfies e e ° h = A. □ 

Remark 4. If /zi + ^ + /i3 = 1, and satisfy the triangle inequality, 

then it can be shown easily that 

4A*l/i2^3 > (A*lA*2 + MlA*3 + M2^3) 2 ^ 3/ii^ 2 ^3, (21) 

i.e., the term inside the of Eq. ((T7|) is positive. 
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Remark 5. For a fixed P°° = e//, what are the possible values of A < that 
make the stochastic matrix P embeddable? In [5J Theorem 3.7], P. Carette puts 
the above question and shows that there exists A < such that P is imbeddablc 
if and only if A ^ A < 0. As a consequence of Theorem 12.61 the exact value of 
A is that 

exp | — ^7= | , when , , satisfy the triangle inequality, 



- exp | — t | , otherwise, 

(22) 

Wheie h = (MiM2+MiM3+ 3 M2M3) a _ l > m = min {^' 



2.1 Proof of the propositions 

Lemma 2.7. IfP satisfies Eq.(l), then its right eigenvectors with eigenvalue A 
span the orthogonal complement of fj,. 

Proof. If / is a right eigenvector of P with eigenvalue A then 

Xf = Pf = eu.'f + A(l - ef//)/ = (1 - Wf + A/, (23) 

so that u 1 f = 0. Hence the eigenvectors span the orthogonal complement of 
/i. □ 

Proof of Proposition \2.4\ The necessity. Since A < and P = e hQ , Q has 
complex eigenvalues 9 and 8 such that e = A. It follows from Lemma [2TT1 that 
Q has fx'Q = 0. 

The sufficiency. Since P satisfies Eq.((T]), one obtains that 

P = Fdiagjl, A, A} F -1 , (24) 

where F = [e, tpi, ip2\, <pi, <P2 are any two linear independent vectors in the 
orthogonal complement of p, by Lemma 12.71 
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Denote by / + ig the eigenvector of Q with eigenvalue 8 = p + ig, q =/= 
0. Clearly /, g are linear independent. u!Q = implies that /, g span the 
orthogonal complement of u.. 

Let P h = e hQ . Hence e hQ = J2n=o ^jQ" im p!ics that if fx'Q = then 
fj/Ph = m' an d that 

Ph(f + ig) = e eh (f + ig) = X(f + ig). (25) 

Hence Phf = A/, Phg = At/. Any transition rate matrix Q has Qe = 0, so that 
P^e = e. Thus one obtains that 

P h = [e, f, g] diag{l, A, A} [e, /, g]-\ (26) 

Note that in Eq.(J2U), one can choose that the matrix F = [e, /, g], which 
implies that = P. □ 

Lemma 2.8. Suppose that f(x) = x + ^ with < x ^ c, where a, c > are 
two constants. Then the minimum of fix) is 



(27) 



(28) 



2y / a, when ^fa ^ c, 
c + -, w/ien y^a > c. 
TTie minimum is attained at the point 

x = \/a, when ^fa ^ c, 
X = c, w/ien \fa > c. 
Proof. It is trivial. □ 

Denote that 

V = {(xi, £2,2:3) <G M 3 : xi, x 2 , x 3 ^ 0,xi + x 2 , x 2 + x 3 , x 3 + xi > 0} , (29) 

£1 =Vn{(x 1 ,x 2 ,x 3 ) eR 3 : Xl ^x 2 ^x 3 }. (30) 
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Since F(x±, x 2 , £3) = F(rx±, rx2, TX3), Vr > 0, the existence of the maximum of 
F(xi, X2, X3) on T> is equal to the existence on the unit sphere which holds true 
since the unit sphere is compact P 

Lemma 2.9. Restricted on £\, the maximum of F(x±, X2, X3) is 



WT\ , when -L > -L + JL 

4(l-/ii) ' (11 ^ ^ 1 



, 1 , i 1 , 1 v , , when —<— + —. 
(— +— + — r Mi Ma 

v Ml M2 M3 ' 



(31) 



The maximum is attained at the points 



1 I -i 

si = ^2 = ^J_l^_l ^3, when — > - + — , 

Ml M2 M3 ^ ^ ^ 

x\ = X2 — Xs, when — < — + — . 

1 Z J ' Ml M2 M3 



(32) 



Proof. Restricted on £\, i.e., xi X2 ^ 2:3, we obtain that 

p, s _ xix 2 + x 2 x 3 + x 3 xi + x\ _ (xi + x 2 )(x 3 + Xi) 

^Xl j X2 , X3 J Xi+3:2 I 1'2+I3 I l'3+Il ^2 _ ^ 3:1+3:2 I 3:2+3:3 I 3:3+3:1 p ' V / 

V Hi ' JL*2 ' H3 ' ^ H\ H2 HS ' 

Let r = xi + X2, s = x 2 + x 3 , t = X3 + x\. Then < r ^ t ^ s, and 

F( Xl ,x 2 ,x 3 ) = -— -g—- 4 G(r,8,t) < G(r,t,t). (34) 

Ml M2 M3 ' 

That is to say, G(r, s, t) attains its maximum when s = t since it is a decreasing 
function of s . Let w = \fj- Then < w ^ 1 and we have that 

G(r,t,t) 



L Ml V M2 Ha' 



2 

M2 1 M: 
Ml ]2 



w + A ii(- + -)- 



= LH. (35) 



It follows from Lemma 12.81 that when — t- — ) > 1, the maximum of 

1 1 ^2 M3 



L(ui) is . j_ j_. 2 , and when — I — — ) 1, the maximum of L(w) is 

' Ml M2 M3 ' ^ 2 ^ 3 

4 f_i^_!_^l ■ In addition, the direct computation yields the maximum points (|3"2")) 

^ M2 M3 ' 

from Lemma T2. 81 □ 



5 The points (1, 0, 0), (0, 1, 0), (0, 0, 1) are not in T>, but the values of F(x\ ,X2,x%) are zero 
at these points, which do not alter the maximum of F(xi, X2, X'j)- 



Imbedding Problem for Three-state Markov Chains 12 

Proof of Proposition ] 2. 51 In order to prove Proposition 2.5 we first assume that 
Hi satisfies the triangular inequality. There is no loss of generality in assuming 
that X\ ^ X2 ^ X3 because if this is not the case, the arguments can be permuted 
to satisfy the restriction. This will leave the value of F invariant if also /Uj are 
permuted correspondingly, but the triangular inequality condition is invariant 
to permutations of /i,-. Thus from Lemma 12.91 we get that F is bounded by the 
restricted maximum: 



1 



F(x 1 ,x 2 ,x 3 ) < — ■ — j — ^_ ■ (36) 



If, however, does not satisfy the triangular inequality, then, without loss of 
generality we can assume that 

+ i (37) 

Ml M2 M3 

Now on the set £ \ we can apply Lemma [2.9l and find that the function is bounded 
by the unrestricted maximum 

cv -\ ^ M1M2M3 

F(x 1 ,x 2 ,x 3 ) < — -. 38) 

4(1 - Mi) 

Now when — > — + — , we have /ii < M2M3 < min(/i2, /13), so that Ui = m = 
min(/ii, M3) and 



77/ \ ^ M1M2M3 . M1M2M3 , qn x 

F(*i,* a ,*a) < ^TT^ < iTT^Tj ■ ( 39 ) 



□ 
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3 The imbedding problem for 3-order transition 
matrix with positive eigenvalues or complex 
eigenvalues 

Proposition 3.1. Suppose P be an indecomposable 3x3 transition probability 
matrix with eigenvalues {1, Ai, A2}. Then it satisfies that 

P 2 - (Ai + A 2 )P + A1A2I = (Ai - 1)(A 2 - 1)P°°. (40) 

It is exactly the Eq.(1.15) of [T3]- Johansen S. points out that the condi- 
tion of the imbedding problem can be given in terms of P and P°° (or u) in 
[IB] . The following two propositions are the direct corollary of Proposition 1.2, 
Proposition 1.4 of and Eg. (HP)) . 

Proposition 3.2. Let P be an indecomposable 3x3 transition probability matrix 
with positive eigenvalues {1, Ai, A2}. P can be imbedded if and only if 

(A 2 - l)logAi - (Ai - l)logA 2 . , . , . 

Pij>H ; t ] — r » iT 3- (41) 

log A 2 - log Ai 



If Ai = A 2 = A, then the right hand side of is in the sense of limit, i.e., fix 
the value of X±, and let A 2 — > Ai, one has that 

Pij ^ Hj ;(A log A - A + 1), i ^ j. (42) 

Proposition 3.3. Let P be an indecomposable 3x3 transition probability matrix 
with complex eigenvalues {1, Ai, A 2 } 7 Ai = re 1 ® , A 2 = re~ 10 , 9 £ (0, n). P can 
be imbedded if and only if 



Pij ^ fJ>j(l - rcos9+ ^-rlogr), i^j. (43) 
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or 

sin 9 

Pij < (J>j(l-rcos9+ _ rlogr), i^j. (44) 

The two propositions appear implicitly in [13j and seem neater than Eq.(l.ll- 
1.13,1.17) of Reference Q3]. 

Remark 6. If P is reversible, then — = — . One should only test half number 
of the inequalities. 

4 Conclusion 

In the present paper, we solve the imbedding problem for 3-order transition 
matrix with coinciding negative eigenvalues by means of an alternate parame- 
terization of the transition rate matrix, which is different from the traditional 
way to calculate the matrix logarithm or the matrix square root. 
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